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1.  Introduction 


Certain  digital  cameras  do  not  produce  the  quality  of  photos  that  are  desired.  A  way 
to  circumvent  this  problem  without  replacing  the  camera  is  through  an  image 
processing  algorithm  (i.e.,  super-resolution  image  reconstruction  [SRIR]).  The 
algorithm  we  are  using  takes  an  input  image  sequence  that  comprises  low- 
resolution  images.  A  correlation  method  is  used  to  estimate  subpixel  shifts  between 
each  low-resolution  aliased  image  with  respect  to  a  reference  image.  An  error- 
energy  reduction  algorithm  is  derived  to  reconstruct  the  high-resolution  alias-free 
output  image.1 

Digital  images  are  represented  by  numerical  values  that  hold  information  about 
each  pixel.  For  color  images,  each  pixel  is  represented  by  a  tuple.  This  means  that 
the  whole  image  can  be  visualized  as  a  cube,  or  an  n-by-3  matrix.  These  numerical 
values  each  represent  a  different  coordinate  of  the  image,  and  these  coordinates 
vary  between  different  color  spaces.  A  color  space  is  a  specific  way  to  organize 
colors.  Color  spaces  can  be  divided  into  3  categories.  The  first  category 
encapsulates  color  spaces  that  describe  color  by  additive  color  principle,  such  as 
the  red,  green,  blue  (RGB)  and  XYZ  color  spaces.  The  second  category  of  color 
spaces  comes  from  conventional  television  signal  standards.  Color  spaces  in  this 
category  include  YCbCr  and  YIQ,  which  was  formerly  used  in  National  Television 
Standards  Committee  (NTSC)  broadcasts.  The  third  category  revolves  around  the 
theory  that  human  eyes  often  perceive  color  in  3  dimensions:  hue,  saturation,  and 
colorfulness.  Within  this  category  are  the  hue-saturation-intensity  (HSI)  and  hue- 
saturation-value  (HSV)  color  spaces,  where  the  I  and  V  represent  intensity  and 
value,  respectively. 

Digital  grayscale  images  are  represented  by  an  n-by-m  matrix.  This  can  be 
visualized  as  a  flat  rectangle,  where  the  numerical  values  represent  only  the 
lightness  of  the  images  and  there  is  no  color  information.  Our  super-resolution 
algorithm  was  designed  to  process  grayscale  images.  However,  we  would  like  to 
be  able  to  super-resolve  color  images,  as  well.  There  are  2  different  ways  to  super¬ 
resolve  color  images.  The  first  way  is  to  take  all  3  “bands”  of  a  color  image 
sequence  and  super-resolve  them  individually,  which  can  work  in  any  color  space. 
However,  this  takes  lots  of  time  and  processing.  The  proposed  approach  is  to  take 
an  image  in  a  color  space  of  the  3  categories  discussed,  where  one  band  represents 
luminance  or  lightness  while  the  other  2  represent  chrominance  or  color,  then  only 
super-resolve  the  luminance  band  sequence.2  The  chrominance  band  sequence  need 
only  to  be  upsampled.  These  3  transformed  bands  can  then  be  combined  to  produce 
a  final,  super-resolved  color  image. 
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This  report  describes  the  5  color  spaces  in  which  we  implemented  this  approach. 
We  then  transformed  our  digital  images  in  the  default  standard  RGB  (sRGB)  color 
space  to  the  desired  color  space  by  employing  several  different  color  transformation 
equations.  In  this  report,  we  detail  each  of  the  color  spaces  we  used  and  the 
transformations.  We  then  describe  the  procedure  of  super-resolution  for  color 
imagery  in  more  depth.  After  that,  we  present  our  results  and  findings. 

Our  input  digital  images  are  in  the  default  color  space  of  sRGB  which  is  a  particular 
RGB  color  space  that  is  defined  by  the  3  chromaticities  of  the  red,  green,  and  blue 
additive  primaries,  as  well  as  a  white  point.  The  sRGB  color  space  is  an  RGB  color 
space  created  by  HP  and  Microsoft  in  1996  for  use  on  monitors  and  the  Internet.  It 
was  standardized  by  the  International  Electrotechnical  Commission  in  1999. 

We  started  by  implementing  the  direct  super-resolution  approach  in  the  sRGB  color 
space,  by  super-resolving  each  band  separately.  Then,  we  implemented  the 
proposed  approach,  in  the  color  spaces  the  CIE  1976  (L*,  a*,  b*)  color  space 
(CIELAB),  YCbCr,  YIQ,  HSV,  and  HSI.  We  chose  these  color  spaces  for  2 
reasons:  ease  of  transformation  to  and  from  sRGB,  and  having  1  luminance  band 
and  2  chrominance  bands. 

2.  Color  Spaces 


2.1  CIELAB 

The  first  color  space  we  looked  at  is  CIELAB.  The  CIELAB  color  space  was 
designed  to  approximate  human  vision  and  describes  mathematically  all 
perceivable  colors  in  the  3  dimensions:  L  for  lightness,  and  a  and  b  for  the  color 
opponents  green-red  and  blue-yellow,  respectively.  Figure  1  illustrates  the 
transformation  equations  and  processes  to  transform  from  sRGB  to  CIELAB.  As 
shown  in  Fig.  1,  there  is  no  simple  transformation  from  sRGB  to  CIELAB,  so  to 
obtain  a  digital  image  in  the  CIELAB  color  space,  we  had  to  first  transform  the 
image  in  sRGB  to  the  XYZ  color  space.  This  simply  involves  multiplying  our 
sRGB  “cube”  by  a  matrix,  and  then  normalizing  for  a  white  point.3  The 
transformation  from  XYZ  to  CIELAB  is  not  as  simple  and  involves  several 
different  mathematical  equations. 
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Step  1:  RGB  to  XYZ 
MAT  = 

0.4125  0.3576  0.1804 
0.2127  0.7152  0.0722 
0.0193  0.1192  0.9502 
XYZ  =  MAT  *  RGB; 


Step  2:  XYZ  to  Lab 

£  =  0.008856 
y  =  903.3 


fx 


f  vs 

if  X  >  £ 

—  jyje+16 
v  Ll6 

otherwise 

[  Ty 

if  V  >  e 

~  jyr+16 
^  116 

otherwise 

( 

if  Z  >  £ 

~  WzF16 

V  116 

otherwise 

Step  3 :  Get  Lab  Values 

L  =  116fy  -  16 

a  =  S00 (fx  -  fy) 

^  =  200(4-4) 


Fig.  1  Transformation  from  sRGB  to  CIELAB 


2.2  YCbCr _ 

The  second  color  space  we  used  is  YCbCr.  The  Y  component  represents  luminance, 
the  Cb  component  represents  the  blue-difference,  and  the  Cr  component  represents 
the  red-difference.  This  color  space  is  defined  by  a  mathematical  coordinate 
transformation  (Fig.  2)  from  an  associated  RGB  color  space,  and  thus  the 
transformation  is  simply  matrix  multiplication. 


Forward  Transformation  Matrix 

Inverse  Transformation  Matrix 

65.4810  128.5530 

24.9660 

0.0046 

0 

0.0063 

-37.7970  -74.2030 

112.0000 

0.0046 

-0.0015 

-0.0032 

112.0000  -93.7860 

-18.2140 

0.0046 

0.0079 

0 

Fig.  2  YCbCr  mathematical  coordinate  transformation 


2.3  YIQ _ 

The  third  color  space,  YIQ,  is  similar  in  that  the  transformation  is  matrix 
multiplication.  YIQ  is  the  color  space  used  by  the  NTSC  color  TV  system.  The  Y 
component  represents  luminance,  while  the  I  and  Q  components  represent  the 
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chrominance  components.4  Figure  3  shows  the  mathematical  coordinate 
transformation  for  YIQ. 


Forward  Transformation  Matrix 

Inverse  Transformation  Matrix 

1.0000 

1.0000  1.0000 

1.0000 

0.9560  0.6210 

0.9560 

0.2720  -1.1060 

1.0000 

-0.2720  -0.6470 

0.6210 

-0.6470  1.7030 

1.0000 

-1.1060  1.7030 

Fig.  3  YIQ  mathematical  coordinate  transformation 

2.4  HSV  and  HSI 


The  fourth  and  fifth  color  spaces  we  used,  HSV  and  HSI,  are  similar.  Both  of  these 
color  spaces  are  common  cylindrical-coordinate  representations  of  points  in  an 
RGB  color  model.  Figure  4  shows  the  visualization  of  the  transformation  as  a 
rearrangement  of  Cartesian  (cube)  RGB  model  to  a  cone  model.  The 
transformations  from  sRGB  to  these  spaces  require  several  different  mathematical 
equations,  as  opposed  to  a  simple  matrix  multiplication. 


White 


Black 


Fig.  4  HSI  biconal  color  model 
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3.  Approaches  of  Super-Resolution  for  Color  Imagery 


3.1  Direct  Approach 


Figure  5  shows  the  direct  approach  of  super-resolution  for  color  imagery  in  which 
a  digital  image  sequence  is  split  into  its  3  separate  bands.  A  digital  image  is 
represented  as  a  cube,  or  an  n-by-m-by-3  matrix.  When  one  splits  the  cube,  one 
then  has  3  n-by-m  matrices.  Each  of  these  displays  as  a  grayscale  image,  since  they 
all  represent  only  one  coordinate  of  the  color  space.  In  our  case,  we  only 
implemented  this  approach  in  the  sRGB  color  space,  so  the  3  coordinates  were  red, 
green,  and  blue.  Then  the  grayscale  image  sequences  are  super-resolved  and 
reassembled  into  the  cube  to  produce  a  super-resolved  color  image. 


Color  Image 
Sequence  in 
RGB  color 
space 


Reconstructed, 
Super-Resolved 
Color  Image 


Fig.  5  Direct  approach  to  super-resolution  for  color  imagery 

3.2  Proposed  Approach 

Figure  6  shows  the  diagram  of  the  proposed  approach  to  super-resolving  color 
images  in  which  only  super-resolving  one  of  the  grayscale  images/coordinates/ 
bands.  The  first  step  to  doing  this  is  taking  our  digital  image  in  the  sRGB  color 
space  and  transforming  it  to  1  of  our  5  desired  color  spaces.  From  there,  we  again 
split  up  the  cube  into  3  n-by-m  matrices.  One  of  these  matrices  represents  the 
image’s  luminance  information,  while  the  other  2  represent  the  chrominance 
information.  The  luminance  sequence  is  super-resolved  while  the  first  frames  of  the 
2  chrominance  sequences  are  upsampled  using  the  US  Army  Research  Laboratory’s 
(ARL)  alias-free  image  upsampling  algorithm.5  Because  not  each  of  the  color 
transformation  matrices  are  normalized  transformations,  after  these  processes,  each 
resultant  output  image  needs  to  be  normalized  to  the  same  boundary  points  that  they 
possessed  before  the  super-resolution  or  upsampling.  This  rescaling  is  to  minimize 
color  distortion  in  our  resultant  image.  After  rescaling,  the  3  output  resultant  images 
are  reconstructed  into  the  cube.  The  cube  then  has  a  reverse  transformation  applied 
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to  it  that  brings  the  super-resolved  image  back  into  the  sRGB  color  space.  We  can 
then  view  the  super-resolved  color  image. 


Color  Image 
Sequence  In  RGB 
Color  Space 


Reconstructed, 
Super-Resolved 
Image  in  RGB 
Color  Space 

Fig.  6  Proposed  approach  to  super-resolution  for  color  imagery 


3.3  Normalization 


The  super-resolution  and  upsampling  processes  could  lead  to  the  scale  of  the 
individual  components  being  altered.  To  successfully  reconstruct  the  super- 
resolved  color  image  while  minimizing  color  distortion,  we  need  to  normalize  the 
images  so  that  the  scales  are  the  same  as  what  they  were  before  super-resolution/ 
upsampling.  We  used  a  simple  equation  to  obtain  the  scaled  image  Is  as  follows: 


j  _  [  G  Ip^nax  \  f  j  _  j  \  .  j 

*s  ~  \  ,  I  *  \li,max  H.min)  it 

Vo.max  lo,minJ 


(i) 


where  I0  is  the  super-resolved  or  upsampled  image,  It  is  the  first  frame  of  the  low- 
resolution  luminance/chrominance  sequence,  and  their  minimums  and  maximums 
are  their  absolute  minimum  and  maximum  values. 


4.  Results 


To  examine  the  merit  of  super-resolution  for  color  image,  we  presented  our  results 
using  2  separate  images,  one  with  a  purple  background  and  one  with  a  green 
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background,  shown  in  Figs.  7  and  8.  Between  the  different  color  spaces,  there  were 
different  requirements  for  normalization,  which  are  detailed  within  the  next  section. 


Fig.  7  An  image  with  a  purple  background 


Fig.  8  An  image  with  a  green  background 


4.1.  Direct  Approach  with  the  Purple  Image 


Pictured  below  in  Fig.  9  is  the  result  of  the  direct  approach.  As  one  can  see,  there 
is  extreme  color  distortion  from  the  original  image  to  the  resultant  image.  Not  only 
is  this  method  inefficient,  but  it  does  not  produce  a  good  result. 
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Super-resolved  R  band 


SRIR 


idprviM 


Low-resolution 


Super-resolved  G  band 


Super-resolved 


W  •  '  i  \ .  *  >  V  » 


SRIR 


Super-resolved  B  band 


Fig.  9  Direct  approach  with  the  purple  image 


4.2  Image  with  Purple  Background 


4.2.1  CIELAB 


For  the  super-resolution  of  the  color  image  sequence  with  a  purple  background  in 
the  CIELAB  color  space,  Fig.  10  shows  that  the  resultant  image’s  purple 
background  is  a  slightly  lighter  shade  of  purple.  Also,  there  is  some  purple  coloring 
on  the  white  paper.  Overall,  the  resultant  image  is  good.  After  the  super-resolution 
and  upsampling  processes,  only  the  super-resolved  component  needed  to  be 
normalized  before  reconstruction. 


Fig.  10  Super-resolution  of  the  purple-color  imagery  in  CIELAB  color  space 
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After  the  transformation  of  the  sRGB  image  to  the  CIELAB  image,  the  numerical 
values  that  represent  the  information  that  make  up  the  digital  image  are  rescaled 
based  on  the  transformation  function.  While  the  input  image  scale  is  0-255,  in 
Fig.  11,  one  can  see  that  the  L,  a,  and  b  components  have  very  different  scales. 


L  component  a  component  b  component 


Fig.  11  Histograms  of  L,  a,  and  b  components,  respectively,  of  the  first  frame  of  the  low- 
resolution  input  sequence  before  the  super-resolution  and  upsampling  processes 

As  shown  in  Figs.  1 1  and  12,  only  the  scale  of  the  L  component  changes  after  the 
super-resolution  process.  This  means  that  we  have  to  normalize  this  component 
back  to  its  original  scale  before  the  super-resolution.  After  doing  this,  we  are  able 
to  successfully  reconstruct  our  super-resolved  color  image. 


Fig.  12  Histograms  of  super-resolved  L  component,  upsampled  a  component,  and 
upsampled  b  component,  respectively 

4.2.2  YCbCr 

For  the  super-resolution  of  the  color  image  with  a  purple  background  in  the  YCbCr 
color  space,  Fig.  13  shows  that  the  resultant  image’s  purple  background  is  again,  a 
slightly  lighter  shade  of  purple.  Again,  there  is  some  purple  coloring  on  the  white 
paper.  Overall,  the  resultant  image  is  good  and  quite  similar  to  the  CIELAB 
resultant  image.  After  the  super-resolution  and  upsampling  processes,  all 
components  needed  to  be  normalized  before  reconstruction. 
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Fig.  13  Super-resolution  of  the  purple-color  imagery  in  YCbCr  color  space 


As  shown  in  Figs.  14  and  15,  neither  super-resolution  nor  upsampling  affect  the 
scale  of  the  images,  so  none  of  them  need  to  be  normalized. 


Y  component  Cb  component  Cr  component 


Fig.  14  Histograms  of  Y,  Cb,  and  Cr  components,  respectively,  of  the  first  frame  of  the  low- 
resolution  input  sequence  before  the  super-resolution  and  upsampling  processes 


Super-resolved  Y  component 


Up-sampled  Cb  component  Up-sampled  Cr  component 


Fig.  15  Histograms  of  super-resolved  Y  component,  upsampled  Cb  component,  and 
upsampled  Cr  component,  respectively 
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4.2.3  YIQ 


For  the  super-resolution  of  the  color  image  with  a  purple  background  in  the  YIQ 
color  space,  Fig.  16  shows  that,  like  with  the  2  previous  color  spaces,  the  resultant 
image’s  purple  background  is  a  slightly  lighter  shade  of  purple.  There  is  slightly 
less  purple  coloring  on  the  white  paper  than  with  the  other  2  color  spaces,  and  the 
black  font  is  slightly  lighter  than  it  was  in  the  previous  2  color  spaces.  After  the 
super-resolution  and  upsampling  processes,  only  the  Y  component  needed  to  be 
normalized  before  reconstruction. 


Fig.  16  Super-resolution  of  the  purple-color  imagery  in  YIQ  color  space 


As  shown  in  Figs.  17  and  18,  only  the  Y  component  needs  to  be  normalized  after 
the  super-resolution  process. 


Y  component 


I  component 


Q  component 


Fig.  17  Histograms  of  Y,  I,  and  Q  components,  respectively,  of  the  first  frame  of  the  low- 
resolution  input  sequence  before  the  super-resolution  and  upsampling  processes 
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Super-resolved  Y  component  Up-sampled  I  component  Up-sampled  Q  component 

Fig.  18  Histograms  of  super-resolved  Y  component,  upsampled  I  component,  and 
upsampled  Q  component,  respectively 

4.2.4  HSV 

For  the  super-resolution  of  the  color  image  with  a  purple  background  in  the  HSV 
color  space,  Fig.  19  shows  that  the  purple  background  is  also  slightly  lighter.  This 
leads  up  to  believe  that  the  super-resolution/upsampling  algorithms  led  to  some 
color  distortion.  There  is  even  less  purple  coloring  on  the  white  paper  in  this  color 
space.  After  the  super-resolution  and  upsampling  processes,  only  the  super- 
resolved  component  needed  to  be  normalized  before  reconstruction. 


Fig.  19  Super-resolution  of  the  purple-color  imagery  in  HSV  color  space 

As  shown  in  Figs.  20  and  21,  only  the  V  component  needed  to  be  rescaled  after 
super-resolution. 
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H  component  S  component  V  component 


Fig.  20  Histograms  of  H,  S,  and  V  components,  respectively,  of  the  first  frame  of  the  low- 
resolution  input  sequence  before  the  super-resolution  and  upsampling  processes 


Up-sampled  H  component  Up-sampled  S  component  Super-resolved  V  component 

Fig.  21  Histograms  of  upsampled  H  component,  upsampled  S  component,  and  super- 
resolved  V  component,  respectively 

4.2.5  HSI 

For  the  super-resolution  of  the  color  image  with  a  purple  background  in  the  HSI 
color  space,  Fig.  22  shows  that  the  resultant  image’s  purple  background  is  a  slightly 
lighter  shade  of  purple  than  the  resultant  image  in  the  HSV  color  space.  However, 
the  color  of  the  white  paper  is  better  preserved  throughout  the  super-resolution  and 
upsampling  processes.  After  the  super-resolution  and  upsampling  processes,  only 
the  super-resolved  component  needed  to  be  normalized  before  reconstruction. 
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Fig.  22  Super-resolution  of  the  purple-color  imagery  in  HSI  color  space 

As  shown  in  Figs.  23  and  24,  only  the  I  component  needed  to  be  rescaled  after 
super-resolution. 


H  component  S  component  I  component 

Fig.  23  Histograms  of  H,  S,  and  I  components,  respectively,  of  the  first  frame  of  the  low- 
resolution  input  sequence  before  the  super-resolution  and  upsampling  processes 


Up-sampled  H  component  Up-sampled  S  component  Super-resolved  I  component 


Fig.  24  Histograms  of  upsampled  H  component,  upsampled  S  component,  and  super- 
resolved  I  component,  respectively 
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4.3  Image  with  Green  Background 


4.3.1  CIELAB 


For  the  super-resolution  of  the  color  image  with  a  green  background  in  the  CIELAB 
color  space,  Fig.  25  shows  that  the  resultant  image’s  green  background  is  a  slightly 
lighter  shade  of  green.  In  this  different  image,  we  can  conclude  that  there  must  be 
some  color  distortion  that  occurs  during  the  super-resolution  and/or  upsampling 
processes.  In  contrast,  the  “white”  paper  has  acquired  a  gray  tint.  In  this  image,  the 
super-resolution  is  not  needed  to  enhance  the  image  quality  as  much  as  in  the 
previous  image.  However,  we  included  it  to  present  the  merit  of  super-resolution  of 
a  variety  of  color  imagery.  After  the  super-resolution  and  upsampling  processes, 
only  the  super-resolved  component  needed  to  be  normalized  before  reconstruction. 


Low-resolution 


i /ersalys'*  * 


Super-resolved 


Fig.  25  Super-resolution  of  the  green-color  imagery  in  Lab  color  space 


As  shown  in  Figs.  26  and  27,  only  the  L  component  needed  to  be  rescaled  after 
super-resolution. 


L  component  a  component  b  component 


Fig.  26  Histograms  of  L,  a,  and  b  components,  respectively,  of  the  first  frame  of  the  low- 
resolution  input  sequence  before  the  super-resolution  and  upsampling  processes 
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Super-resolved  L  component 


Up-sampled  a  component 


Up-sampled  b  component 


Fig.  27  Histograms  of  super-resolved  L  component,  upsampled  a  component,  and 
upsampled  b  component,  respectively 


4.3.2  YCbCr 


For  the  super-resolution  of  the  color  image  with  a  green  background  in  the  YCbCr 
color  space,  Fig.  28  shows  that  the  resultant  image  comes  out  pretty  close  to  the 
original  image,  compared  to  the  CIELAB  color  space’s  resultant  image.  There  is 
less  color  distortion.  After  the  super-resolution  and  upsampling  processes,  only  the 
super-resolved  component  needed  to  be  normalized  before  reconstruction. 


Fig.  28  Super-resolution  of  green  color  imagery  in  YCbCr  color  space 

As  shown  in  Figs.  29  and  30,  only  the  Y  component  needed  to  be  rescaled  after 
super-resolution. 
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Y  component  Cb  component  Cr  component 

Fig.  29  Histograms  of  Y,  Cb,  and  Cr  components,  respectively,  of  the  first  frame  of  the  low- 
resolution  input  sequence  before  the  super-resolution  and  upsampling  processes 


Super-resolved  Y  component 


Up- sampled  Cb  component 


Up-sampled  Cr  component 


Fig.  30  Histograms  of  super-resolved  Y  component,  upsampled  Cb  component,  and 
upsampled  Cr  component,  respectively 


4.3.3  YIQ 

For  the  super-resolution  of  the  color  image  with  a  green  background  in  the  YIQ 
color  space,  Fig.  3 1  shows  that  this  resultant  image  also  comes  out  pretty  close  to 
the  original  image  with  minimal  color  distortion.  After  the  super-resolution  and 
upsampling  processes,  only  the  super-resolved  component  needed  to  be  normalized 
before  reconstruction. 
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Fig.  31  Super-resolution  of  the  green-color  imagery  in  YIQ  color  space 

Figures  32  and  33  show  that  only  the  Y  component  needed  to  be  rescaled  after 
super-resolution. 


Y  component  I  component  Q  component 

Fig.  32  Histograms  of  Y,  I,  and  Q  components,  respectively,  of  the  first  frame  of  the  low- 
resolution  input  sequence  before  the  super-resolution  and  upsampling  processes 


Super-resolved  Y  component 


Up-sampled  I  component  Up-sampled  Q  component 


Fig.  33  Histograms  of  super-resolved  Y  component,  upsampled  I  component,  and 
upsampled  Q  component,  respectively 
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4.3.4  HSV 


In  contrast  to  all  of  the  previous  examples,  this  reconstruction  in  the  HSV  color 
space  produces  not  a  lighter  background,  but  a  background  with  a  seemingly  blue 
tint  over  the  green  background.  This  can  be  seen  in  Fig.  34.  After  the  super¬ 
resolution  and  upsampling  processes,  all  components  needed  to  be  normalized 
before  reconstruction. 


Fig.  34  Super-resolution  of  the  green-color  imagery  in  HSV  color  space 


Figures  35  and  36  show  that  all  3  components  needed  to  be  rescaled  after  super¬ 
resolution  and  upsampling. 


H  component  S  component  V  component 


Fig.  35  Histograms  of  H,  S,  and  V  components,  respectively,  of  the  first  frame  of  the  low- 
resolution  input  sequence  before  the  super-resolution  and  upsampling  processes 
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Fig.  36  Histograms  of  upsampled  H  component,  upsampled  S  component,  and  super- 
resolved  V  component,  respectively 


4.3.5  HSI 

Similarly  to  the  resultant  HSV  reconstructed  image  with  a  green  background,  the 
resultant  image  from  using  the  HSI  color  space  produces  some  color  distortion  over 
the  green  background,  with  a  more  concentrated  blue  tint  seen  in  Fig.  37.  After  the 
super-resolution  and  upscaling  processes,  all  components  needed  to  be  normalized 
before  reconstruction. 


i /ersalys* '  » /ersa!ys> ' 


Fig.  37  Super-resolution  of  the  green-color  imagery  in  HSI  color  space 

Figures  38  and  39  show  that  all  3  components  needed  to  be  rescaled  after  super¬ 
resolution  and  upsampling. 
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H  component  S  component  I  component 


Fig.  38  Histograms  of  H,  S,  and  I  components,  respectively,  of  the  first  frame  of  the  low- 
resolution  input  sequence  before  the  super-resolution  and  upsampling  processes 


Fig.  39  Histograms  of  upsampled  H  component,  upsampled  S  component  and  super- 
resolved  I  component,  respectively 

5.  Discussion  and  Conclusion 


Looking  at  the  histogram  results,  the  super-resolved  images  all  are  scaled  to  be 
0-255.  This  explains  why  most  of  the  color  space  inverse  transformations  require 
the  super-resolved  images  to  be  scaled.  The  ones  that  don’t  require  this  have 
transformations  that  require  the  range  to  be  0-255.  The  upsampling  process  does 
not  change  the  scale  for  the  most  of  the  examples.  However,  it  changes  the  scales 
of  HSI  and  HSV  for  green  images.  Some  further  research  needs  to  be  done  to 
explain  this. 

6.  Recommendations 


Going  forward,  there  are  a  few  things  that  can  be  done  to  elaborate  the  results.  Also, 
there  are  several  more  color  spaces  in  which  the  proposed  approach  could  be 
implemented. 
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List  of  Symbols,  Abbreviations,  and  Acronyms 


ARL 

US  Army  Research  Laboratory 

CIELab 

CIE  1976  (L*,  a*,  b*)  color  space 

HSI 

hue-saturation-intensity 

HSV 

hue-saturation-value 

I 

intensity 

NTSC 

National  Television  System  Committee 

RGB 

red,  green,  and  blue 

sRGB 

standard  red,  green,  and  blue 

SRIR 

super-resolution  image  reconstruction 

V 

value 
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Glossary 


CIELAB 

Color  space  was  designed  to  approximate  human  vision,  and 
describes  mathematically  all  perceivable  colors  in  the  3 
dimensions:  L  for  lightness,  and  a  and  b  for  the  color  opponents 
green-red  and  blue-yellow,  respectively. 

YCbCr 

Y  component  represents  luminance,  the  Cb  component 
represents  the  blue-difference,  and  the  Cr  component  represents 
the  red-difference. 

YIQ 

Y  component  represents  luminance,  while  the  I  and  Q 
components  represent  the  chrominance  components. 
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